Reorganize noauto test suite by dependency groups to resolve conflicts by Copilot · Pull Request #1290 · PyThaiNLP/pythainlp

Copilot · 2026-02-05T14:17:09Z

What do these changes do

Separates the monolithic noauto test suite into 5 dependency-specific test suites: noauto_torch, noauto_tensorflow, noauto_onnx, noauto_cython, and noauto_network. Adds corresponding optional dependency groups in pyproject.toml for isolated installation and testing.

Test structure:

tests.noauto_torch - PyTorch, transformers, attacut (tokenization, transliteration, parsing, summarization, NER/POS)
tests.noauto_tensorflow - TensorFlow, deepcut
tests.noauto_onnx - ONNX Runtime (tokenization, transliteration, POS tagging, spell correction)
tests.noauto_cython - phunspell spell checking
tests.noauto_network - HuggingFace Hub downloads
tests.noauto - Umbrella suite loading all above

Naming convention:

Python modules use underscores: tests.noauto_torch
pip extras use hyphens: pip install "pythainlp[noauto-torch]"

Installation example:

# Install and test torch features only
pip install "pythainlp[noauto-torch]"
python -m unittest tests.noauto_torch

# Or install and test tensorflow features only
pip install "pythainlp[noauto-tensorflow]"
python -m unittest tests.noauto_tensorflow

What was wrong

The monolithic noauto test suite mixed PyTorch and TensorFlow dependencies, causing transitive dependency version conflicts. Installing all dependencies took >1 hour and frequently failed. No mechanism existed to test heavy-dependency features in isolation.

How this fixes it

Dependency isolation: PyTorch and TensorFlow are now in separate test suites with separate optional dependency groups, eliminating version conflicts.

Resource optimization: Install only required dependencies per test group (~200MB-3GB vs ~5GB for all).

CI/CD ready: Each suite can run in separate workflows with appropriate dependency isolation.

Changes:

Created 5 test directories with 9 test files (31 test methods total)
Added 5 optional dependency groups in pyproject.toml: noauto-torch, noauto-tensorflow, noauto-onnx, noauto-cython, noauto-network
Removed legacy monolithic test files
Updated tests/README.md with installation instructions and test suite documentation
Updated CHANGELOG.md

Your checklist for this pull request

Passed code styles and structures
Passed code linting checks and unit test

Original prompt

Assess how far we reach subgoals in #935

The "noauto" test suite is still rely on many heavy dependencies.
Consider break them down by dependency group.

For example, test cases that rely on torch and its ecosystem dependencies, will be grouped in "noauto-torch"; while test cases that rely on tensorflow and its ecosystem dependencies, will be grouped in "noauto-tensorflow".

This will allow us to test "noauto-torch" and "noauto-tensorflow" seperately, and avoid a case that some torch dependencies may have different version requirement from some tensorflow dependecies.

Test cases that rely on the same very large download, may also be considered to be grouped together when possible, to optimize resource.

Follow the standardized test suites separation and naming convention, adapted as appropriate and as consistent.

Create new optional dependency extras group in pyproject.toml as appropriate
(this will be useful for continuous testing in CI, and for mypy type checkers)

Properly document this.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: bact <128572+bact@users.noreply.github.com>

…emove legacy test files Co-authored-by: bact <128572+bact@users.noreply.github.com>

coveralls · 2026-02-05T14:31:29Z

coverage: 65.355% (+0.6%) from 64.791%
when pulling 0cf1aa1 on copilot/refactor-noauto-test-suites
into a498d65 on dev.

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Removed unused import 'word_dict_trie' from test file.

Copilot

Pull request overview

This pull request reorganizes the monolithic noauto test suite into 5 dependency-specific test suites to resolve version conflicts between PyTorch and TensorFlow dependencies.

Changes:

Separated tests into dependency-specific suites: noauto_torch (18 methods), noauto_tensorflow (1 method), noauto_onnx (8 methods), noauto_cython (4 methods), and noauto_network (1 method)
Added 5 optional dependency groups in pyproject.toml for isolated testing
Updated documentation in tests/README.md with installation and usage instructions
Removed legacy monolithic test files from tests/noauto/ directory

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 24 comments.

Show a summary per file

File	Description
tests/noauto_torch/init.py	Module initialization for PyTorch test suite with load_tests protocol
tests/noauto_torch/testn_tokenize_torch.py	Tests for tokenization, transliteration, parsing, summarization, and augmentation requiring torch/transformers
tests/noauto_torch/testn_tag_torch.py	Tests for NER/POS tagging requiring torch/transformers
tests/noauto_torch/testn_spell_torch.py	Tests for spell correction requiring torch/transformers
tests/noauto_tensorflow/init.py	Module initialization for TensorFlow test suite
tests/noauto_tensorflow/testn_tokenize_tensorflow.py	Tests for deepcut tokenizer requiring TensorFlow
tests/noauto_onnx/init.py	Module initialization for ONNX Runtime test suite
tests/noauto_onnx/testn_tokenize_onnx.py	Tests for tokenization, transliteration, POS tagging, and spell correction requiring onnxruntime
tests/noauto_cython/init.py	Module initialization for Cython-based test suite
tests/noauto_cython/testn_spell_cython.py	Tests for phunspell spell checker (moved wanchanberta and HuggingFace tests to appropriate suites)
tests/noauto_network/init.py	Module initialization for network-dependent test suite
tests/noauto_network/testn_spell_network.py	Tests for HuggingFace Hub spell suggestions requiring network access
tests/noauto/init.py	Updated to serve as umbrella suite that includes all specialized test suites
tests/noauto/testn_tokenize.py	Removed - tests migrated to specialized suites
pyproject.toml	Added 5 optional dependency groups (noauto-torch, noauto-tensorflow, noauto-onnx, noauto-cython, noauto-network)
tests/README.md	Comprehensive documentation of new test suite structure with installation and usage instructions
CHANGELOG.md	Added entry for test suite reorganization

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

bact · 2026-02-05T14:55:39Z

@copilot apply changes based on the comments in this thread and continue to add more tests to noauto groups.

- Change pip install commands to use hyphens (noauto-cython, not noauto_cython) in README - Update pyproject.toml comments to use underscores for Python module names (tests.noauto_torch not tests.noauto-torch) - Fix imports in test files to import actual functions not modules: * thai2rom.romanize instead of module thai2rom * KeyBERT class and mT5Summarizer class instead of module references * thai2rom_onnx.romanize instead of module * words_spelling_correction.correct instead of module - Remove duplicate unittest imports in __init__.py files (use 'from unittest import main' in __main__ block) Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot · 2026-02-05T14:58:53Z

@copilot apply changes based on the comments in this thread and continue to add more tests to noauto groups.

Applied all changes from the code review in commit 42d6db2:

Fixed pip install commands to use hyphens (e.g., noauto-cython)
Updated pyproject.toml comments to reference Python modules with underscores (e.g., tests.noauto_torch)
Corrected import statements to import actual functions/classes instead of modules
Removed duplicate unittest imports

The test files now correctly import:

thai2rom.romanize, thaig2p.transliterate, etc. (functions from modules)
KeyBERT and mT5Summarizer classes
thai2rom_onnx.romanize and words_spelling_correction.correct functions

Would you like me to add more tests to specific noauto groups? Currently:

noauto_torch: 18 methods (tokenization, transliteration, parsing, summarization, augmentation)
noauto_onnx: 8 methods (tokenization, transliteration, POS tagging, spell correction)
noauto_tensorflow: 1 method (expandable)
noauto_cython: 4 methods (expandable)
noauto_network: 1 method (expandable)

bact · 2026-02-05T16:04:49Z

@copilot apply changes based on the comments in this thread

- Remove test_dependency_parsing_v2 (function doesn't exist) - Fix KeyBERT.extract_keywords to use max_keywords parameter instead of n - Fix mT5Summarizer.summarize to expect list return type instead of str - Fix augmentation tests to use Thai2transformersAug and ThaiTextAugmenter classes - Fix spell correction test to use get_words_spell_suggestion function Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot · 2026-02-05T16:06:52Z

@copilot apply changes based on the comments in this thread

Applied all changes from the second code review in commit e95a7d2:

Fixed function/class usage:

Removed test_dependency_parsing_v2 (function doesn't exist in pythainlp.parse)
Changed KeyBERT.extract_keywords(text, n=2) → extract_keywords(text, max_keywords=2)
Changed mT5Summarizer.summarize assertion from str → list (correct return type)
Changed augmentation tests to instantiate classes:
- Thai2transformersAug() instead of calling non-existent aug_wangchanberta
- ThaiTextAugmenter() instead of calling non-existent aug_phayathaibert
Changed correct → get_words_spell_suggestion in spell correction test

All test imports now reference actual callable functions/classes with correct method signatures.

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

bact · 2026-02-05T18:21:39Z

@copilot apply suggestions in #1290 (review)

- Remove sentence-transformers from documentation (not in pyproject.toml dependencies) - Fix thai_nner to thai-nner with hyphen to match pyproject.toml naming Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot · 2026-02-05T18:23:14Z

@copilot apply suggestions in #1290 (review)

Applied all suggestions from the third code review in commit 9ae1d7a:

Removed sentence-transformers from documentation in both tests/noauto_torch/__init__.py and tests/README.md (not in pyproject.toml dependencies)
Fixed thai_nner to thai-nner with hyphen to match the actual package name in pyproject.toml

Documentation now accurately reflects the dependencies listed in pyproject.toml lines 197-206.

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

bact · 2026-02-05T21:54:40Z

@copilot dev branch is update. please resync and review the test suites.

Copilot · 2026-02-05T21:56:42Z

@copilot dev branch is update. please resync and review the test suites.

Successfully rebased on dev branch (commit 38543dd). The branch now includes all latest changes from dev including:

Braille module additions (Add pythainlp.braille module for Thai braille conversion #1287)
Translation exclude words (Add exclude_words parameter to translation methods #1216)
Type annotation fixes (Fix type annotation errors and remove unused type:ignore comments #1281)
Various documentation updates

Test suite structure verified:

5 modular test directories created (noauto_torch, noauto_tensorflow, noauto_onnx, noauto_cython, noauto_network)
31 total test methods across all suites (17 torch, 8 onnx, 1 tensorflow, 4 cython, 1 network)
All test files have correct syntax
Tests won't load without dependencies installed (expected behavior)

No conflicts during rebase. All documentation and code review fixes from previous rounds remain intact.

Updated optional dependencies section with headings for better organization.

sonarqubecloud · 2026-02-06T10:50:53Z

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Initial plan

68c18ea

Copilot AI assigned Copilot and bact Feb 5, 2026

Copilot started work on behalf of bact February 5, 2026 14:17 View session

bact added the tests Unit test, code coverage, test case label Feb 5, 2026

Copilot AI and others added 4 commits February 5, 2026 14:22

Create modular noauto test suites by dependency groups

a8ed985

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Add deprecation notices to legacy noauto test files

e74b912

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Address code review comments and add CHANGELOG entry

91523b5

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Fix: Use underscores instead of hyphens in test directory names and r…

e70a20c

…emove legacy test files Co-authored-by: bact <128572+bact@users.noreply.github.com>

Add comprehensive tests for torch and onnx-based functions

6b664a2

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Refactor noauto test suite by dependency groups~~ Reorganize noauto test suite by dependency groups to resolve conflicts Feb 5, 2026

Copilot AI requested a review from bact February 5, 2026 14:39

Copilot finished work on behalf of bact February 5, 2026 14:39

Remove unused import in testn_tokenize_torch.py

e80e99e

Removed unused import 'word_dict_trie' from test file.

bact requested a review from Copilot February 5, 2026 14:43

Copilot started reviewing on behalf of bact February 5, 2026 14:43 View session

Copilot AI reviewed Feb 5, 2026

View reviewed changes

bact and others added 5 commits February 5, 2026 14:53

Update tests/README.md

c6d0b4d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/README.md

d753451

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/README.md

4d087d6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/README.md

dbaa40f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update pyproject.toml

3f51061

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of bact February 5, 2026 14:56 View session

Copilot finished work on behalf of bact February 5, 2026 14:59

bact requested a review from Copilot February 5, 2026 15:03

Copilot started reviewing on behalf of bact February 5, 2026 15:03 View session

Copilot started work on behalf of bact February 5, 2026 16:05 View session

Copilot finished work on behalf of bact February 5, 2026 16:08

bact requested a review from Copilot February 5, 2026 16:53

Copilot started reviewing on behalf of bact February 5, 2026 16:53 View session

Copilot AI reviewed Feb 5, 2026

View reviewed changes

Comment thread tests/noauto_torch/__init__.py Outdated

Comment thread tests/README.md Outdated

Copilot started work on behalf of bact February 5, 2026 18:22 View session

Fix: Update documentation to match actual dependencies

9ae1d7a

- Remove sentence-transformers from documentation (not in pyproject.toml dependencies) - Fix thai_nner to thai-nner with hyphen to match pyproject.toml naming Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot finished work on behalf of bact February 5, 2026 18:24

bact requested a review from Copilot February 5, 2026 18:33

Copilot started reviewing on behalf of bact February 5, 2026 18:34 View session

Copilot AI reviewed Feb 5, 2026

View reviewed changes

Comment thread tests/README.md Outdated

Update tests/README.md

789d54b

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

bact added this to the 5.3 milestone Feb 5, 2026

bact marked this pull request as ready for review February 5, 2026 21:54

Copilot started work on behalf of bact February 5, 2026 21:55 View session

Copilot started work on behalf of bact February 6, 2026 10:31 View session

Copilot finished work on behalf of bact February 6, 2026 10:35

Organize optional dependencies in pyproject.toml

0cf1aa1

Updated optional dependencies section with headings for better organization.

bact merged commit d55e615 into dev Feb 6, 2026
21 of 32 checks passed

bact deleted the copilot/refactor-noauto-test-suites branch February 6, 2026 10:51

bact added the refactoring a technical improvement which does not add any new features or change existing features. label Feb 6, 2026

Uh oh!

Uh oh!

Conversation

Copilot AI commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What do these changes do

What was wrong

How this fixes it

Your checklist for this pull request

Uh oh!

coveralls commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bact commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

bact commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

bact commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

bact commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

sonarqubecloud Bot commented Feb 6, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Feb 5, 2026 •

edited

Loading

coveralls commented Feb 5, 2026 •

edited

Loading